VISIBLE: Video-Assisted Keystroke Inference from Tablet Backside Motion

The deep penetration of tablets in daily life has made them attractive targets for keystroke inference attacks that aim to infer a tablet user’s typed inputs. This paper presents VISIBLE, a novel video-assisted keystroke inference framework to infer a tablet user’s typed inputs from surreptitious video recordings of tablet backside motion. VISIBLE is built upon the observation that the keystrokes on different positions of the tablet’s soft keyboard cause its backside to exhibit different motion patterns. VISIBLE uses complex steerable pyramid decomposition to detect and quantify the subtle motion patterns of the tablet backside induced by a user’s keystrokes, differentiates different motion patterns using a multi-class Support Vector Machine, and refines the inference results using a dictionary and linguistic relationship. Extensive experiments demonstrate the high efficacy of VISIBLE for inferring single keys, words, and sentences. In contrast to previous keystroke inference attacks, VISIBLE does not require the attacker to visually see the tablet user’s input process or install any malware on the tablet.